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Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above Is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )^ Responsive to connmunication(s) filed on 01 November 1301 . 
2a)n This action is FINAL. 2b)^ This action is non-final. 

3) n Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 0,G. 213. 

Disposition of Claims 

4) E Claim(s) 7-74 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) \3 Claim(s) is/are allowed. 

6) K Claim(s) t 4- 9. 11-14 is/are rejected. 

7) ^ Claim{s) 2.3 and 10 is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) n The specification is objected to by the Examiner. 

10)13 The drawing(s) filed on 13 November 2001 is/are: a)n accepted or b)|Sl objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1 , 121(d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 

Priority under 35 U.S.C. § 119 

12)0 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)n All b)n Some * c)^ None of: 

1 .□ Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n , Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached^detailed Office action for a list of the certified copies not received. 


Attachment(s) 

1 ) Notice of References Cited (PTO-892) 4) □ Inten/iew Summary (PTO-41 3) 

2) □ Notice of Draflsperson's Patent Drawing Review (PTO-948) Paper No(s)/Mail Date. . 

3) IS Infomiation Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 5) □ Notice of Infonnal Patent Application (PTO-152) 

Paper No(s)/Mail Date 11/13/01 . 6) □ Other: . 


U.S. Patent and Trademark Office 

PTOL-326 (Rev. 1-04) 
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DETAILED ACTION 
Information Disclosure Statement 

1 . The information disclosure statement (IDS) submitted on 1 1/13/01 is considered 
by the examiner. 

Drawings 

2, The drawings are objected to under 37 CFR 1 .83(a). The drawings must show 
every feature of the invention specified in the claims. Therefore, the three- dimensional 
model and spatial three- dimensional model must be shown or the feature(s) canceled 
from the claim(s). No new matter should be entered. 

Corrected drawing sheets in compliance with 37 CFR 1 .121(d) are required in 
reply to the Office action to avoid abandonment of the application. Any amended 
replacement drawing sheet should include all of the figures appearing on the immediate 
prior version of the sheet, even if only one figure is being amended. The figure or figure 
number of an amended drawing should not be labeled as "amended." If a drawing figure 
is to be canceled, the appropriate figure must be removed from the replacement sheet, 
and where necessary, the remaining figures must be renumbered and appropriate 
changes made to the brief description of the several views of the drawings for 
consistency. Additional replacement sheets may be necessary to show the renumbering 
of the remaining figures. The replacement sheet(s) should be labeled "Replacement 
Sheet" in the page header (as per 37 CFR 1 .84(c)) so as not to obstruct any portion of 
the drawing figures. If the changes are not accepted by the examiner, the applicant will 
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be notified and informed of any required corrective action in the next Office action. The 
objection to the drawings will not be held in abeyance. 

3. Figure 6 should be designated by a legend such as -Prior Art- because only 
that which is old is illustrated. See MPEP § 608.02(g). Corrected drawings in 
compliance with 37 CFR 1 .121(d) are required in reply to the Office action to avoid 
abandonment of the application. The replacement sheet(s) should be labeled 
"Replacement Sheet" in the page header (as per 37 CFR 1 .121(d)) so as not to obstruct 
any portion of the drawing figures. If the changes are not accepted by the examiner, the 
applicant will be notified and informed of any required corrective action in the next Office 
action. The objection to the drawings will not be held in abeyance. 

Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

This application currently names joint inventors. In considering patentability of 
the claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of 
the various claims was commonly owned at the time any inventions covered therein 
were made absent any evidence to the contrary. Applicant is advised of the obligation 
under 37 CFR 1 .56 to point out the inventor and invention dates of each claim that was 
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not commonly owned at the time a later invention was made in order for the examiner to 
consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f) or (g) 
prior art under 35 U.S.C. 1 03(a). 

5. Claims 1, 4,7-9 and 13-14 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over a combination of Sezllski et al. (US 6,044,181) and Huber (automatic 
3D modeling using range images obtained from unknown viewpoints — IDS). 

With regard to claim 1, which is representative of claim 9, Szeliski discloses a 
method for deriving a three-dimensional model of a scene from a plurality of images of 
the scene (figures 1-3), said method comprising: 

(a) generating a plurality of three-dimensional panoramic images of a 
scene (figures 2-3; column 4, lines 45-65); 

(b) determining transformations that align the plurality of three-dimensional 
panoramic images (figures 4-5, also see column 13 line 25 to column 14 line 45); 

(c) integrating spatial information from the plurality of three-dimensional 
panoramic images to form a spatial three-dinlensional model of the scene (figures 1 
and 2B; environment/ texture map memory 270 stores a 3-D model); 

(d) integrating intensity (column 17, lines 12-25) and texture information 
(figure 2B) from the plurality of three-dimensional panoramic images (figure 3) onto the 
spatial three-dimensional model to form a three-dimensional model of the scene (270 in 
figure 2B) containing both spatial and intensity information (figures 11-12). 

Szeliski is silent about range images as claimed. Huber, in the same field of 
endeavor of automatic 3D modeling using range images obtained from unknown 
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viewpoints (title) and same problem solving field of range image information disclose the 
use of range images. In particular, Huber constructs a 3D model from a plurality of 
range images using a surface matching algorithm. It would have been obvious to a 
person of ordinary skill in the art at the time of the invention was made to modify 
Szeliski by using range images in creating 3D panoramic image model as taught by 
Huber because such a modification will allow a system to create an automatically 3D 
model of a scene from a set of range images obtained from unknown viewpoints as 
mentioned Huber in the title and also at least in the abstract. 

As to claim 4, Szeliski discloses the method wherein step (b) further comprises: 

(a) determing one or more pairs of three-dimensional panoramic images 
that contains some common scene information (figures 1, 2B, 3); 

(b) determining the transformations that align each pair of three 
dimensional panoramic images that contain some common scene information (figure 1, 
2B); 

(c) determining global inconsistencies in the transformations found in step 
(b) (column 20, lines 34-63). 

As to claims 7 and 13, Sezliski discloses the method wherein the three- 
dimensional panoramic image Is a color image (column 28, lines 26-50). 

As tom claims 8 and 14, Huber discloses the method wherein one or more range 
images are juxtaposed between a pair of three-dimensional panoramic images (figure 
2). 
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6. Claims 5 and 1 1 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
the combination of Sezliski et al. (US 6,044,181) and Huber (automatic 3D modeling 
using range images obtained from unknown viewpoints— IDS) as applied to claim 1 
above, and further in view of Nayar et al. (US 4,912,336). 

The arguments as to the relevance of the aforesaid combination as 
applied above are incorporated herein. The aforesaid combination does not disclose 
Lambertian reflectance model. Nayar provides this model as shown in figure 2A and 
column 5 line 65 to column 6 line 20. It would have been obvious to one of ordinary skill 
in the art at the time of the invention was made to modify Szeliski and Huberts 3D 
panoramic model generating system by adding a Lambertian reflectance model as 
taught by Nayar because such a modification would have allowed for a system to 
determine the parameters of the reflectance model at each surface point as shown by 
Nayar at column 2, lines 13-15). 

7. Claims 6 and 12 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
a combination of Szeliski et al. (US 6,044,181) and Daniel F. Huber (Automatic 3D 
Modeling Using Range Images obtained From Unknown Viewpoints— IDS ) as applied 
to claims 1 , 4, 7-9 and 13-14 above and further in view of applicant's admitted prior art 
in the specification at page 10, lines 4-14. 

As to claims 6 and 12, Szeliski discloses the method, wherein the step (d) of 
integrating the intensity and texture information from the plurality of three-dimensional 
panoramic images (figure 2) but neither Szeliski nor Huber discloses a reflectance 
model that depends on the viewpoint of the observer. However, applicant's admitted 
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prior art cleariy shows the use of a reflectance nnodel as mentioned at column 10, lines 
4-14. It would have been obvious to use a reflectance model in the method of Szeliski 
as modified by Huber as taught by the admitted prior art. Doing so will yield a more 
accurate rendering that incorporates the differences in surface reflectance as a function 
of the viewing angle. 

Allowable Subject Matter 

8. Claims 2-3 and 10 are objected to as being dependent upon a rejected base 
claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 
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Other prior art cited 

9. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Chen et al. (US 6,677,982 B1 ) disclose a method for three dimensional spatial 
panorama formations. 

Bernardini et al. (US 6,750,873 81) disclose a high quality texture reconstruction 
from multiple scans. 

Shum et al. (US 6,639,596 81) disclose a stereo reconstruction from multi- 
perspective panoramas. 

Debevec (US 6,628,298 81) discloses an apparatus and method for rendering 
synthetic objects into real scenes using measurements of scene illumination. 

Herman et al. (US 6,075,905) disclose a method and apparatus for mosaic image 
construction. 

Zhang et al. (US 6,639,594 82) disclose a view-dependent image synthesis. 
Raskar et al. (US 6,677,956 82) disclose a method for cross-images of 
intensities of multiple images of a scene for seamless reconstruction. 
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Contact Information 


10. Any inquiry concerning this connmunication or earlier communications from the 
examiner should be directed to Sheela C Chawan whose telephone number is 703- 
305- 4876. The examiner can normally be reached on Monday - Thursday 8 - 6.30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Bhavesh Mehta can be reached on 703-308-5246. The fax phone number 
for the organization where this application or proceeding is assigned is 703-872-9306, 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more infomiation about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-21 7-91 97 (toll-free). 


Sheela Chawan 
Patent Examiner 
Group Art Unit 2625 
Jan 21,2005 



